Hybrid control device

ABSTRACT

A hybrid control system for a robot can include a neuronal control portion and a non-neuronal control portion.

CLAIM OF PRIORITY

This application claims priority to U.S. Provisional Application No.60/716,821 entitled “Hybrid Control Device” filed Sep. 13, 2005, whichis incorporated herein by reference.

BACKGROUND OF INVENTION

The present invention concerns robots, especially robots adapted tolearn about their environment.

Robots are becoming more and more popular. One class of robots that haveparticular interest is robots that use artificial intelligence to learnto interact with its environment.

SUMMARY OF INVENTION

Embodiments of the present invention concern Hybrid Control Systems thatcan consist of both neural and non-neural control elements.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a hybrid control device.

FIGS. 2A-2B shows a first example of a hybrid control device.

FIGS. 3A-3C shows a second example of a hybrid control device.

FIG. 4 shows a hybrid control device on a separate robot platform.

FIG. 5 shows an example of an image processing for a hybrid controldevice.

FIG. 6 shows an example of hybrid control device operation.

FIG. 7 shows finite state automata showing high level play behavior forone example of a hybrid device.

DETAILED DESCRIPTION

Aspects of the invention are illustrated by way of example and not byway of limitation in the figures of the accompanying drawings in whichlike references indicate similar elements. It should be noted thatreferences to “an”, “one” and “various” embodiments in this disclosureare not necessarily to the same embodiment, and such references mean atleast one. In the following description, numerous specific details areset forth to provide a thorough description of the invention. However,it will be apparent to one skilled in the art that the invention may bepracticed without these specific details. In other instances well-knownfeatures have not been described in detail so as not to obscure theinvention.

Hybrid Control Systems can consist of both neural and non-neural controlelements. In one embodiment, a device is guided by a hybrid controlsystem, comprising of a neuronal control portion, such as a simulatednervous system, and a non-neuronal control portion, such ascomputational algorithms, artificial intelligence, or control systemmethods. The neuronal control portion can be based on a simplified modelof vertebrate nervous system. For this reason, devices under the controlof the neuronal control portion can be called brain-based devicesinstead of robots.

FIG. 1 shows a device including sensors 104, 106 and 108, and acontroller 110 receiving inputs from the sensors 104, 106 and 108. Thecontroller 110 can include a neuronal controller portion 112 and anon-neuronal controller portion 114. Actuators 116, 118 and 120 canreceive control commands from the controller 110.

The sensors 104, 106 and 108 can be any type of sensor including lasersensors, IR sensors, odometer sensors, such as wheel sensors, tactilesensors, cameras, and the like. The actuators can be any type ofactuator including locomotion actuators such as wheel motors, steeringmotors, and actuators that interact with other objects such as grabbers,cameras orientation motors and the like.

In one embodiment, at least one of the actuators 116 is controlled bythe neuronal controller portion 112. At least another one of theactuators 120 can be controlled by the non-neuronal controller portion114.

In one embodiment, an actuator 118 can be controlled by both a neuronalcontroller portion 112 and a non-neuronal controller portion 114.

The device 102 can be a robot. Both the neuronal controller portion 112and the non-neuronal controller portion 114 can be located on board thedevice, such as a robot. Alternately, some portion of the controller canbe remote from the device and connected to the device with a link, suchas a wireless link.

The neuronal controller portion 112 can have a number of neuronal units.The neuronal units can emulate neurons or groups of neurons.

In one embodiment, a neuronal unit can be simulated by a mean firingrate model. The mean firing rate variable of each neuronal unit cancorrespond to the average activity of a group of about 100 real neuronsduring a time period of about 30 ms.

In one embodiment, there are two types of connections in the model,voltage-independent (VI) and voltage-dependent (VD). VI connections onlyinject current into a neuronal unit that already has activity. VDconnections inject current regardless of the post synaptic activity.

The pre-synaptic input due to VI connections to the activity of neuronalunit i can be given by ?(t) = ??c_(ij)s_(j)(t)?indicates text missing or illegible when filedwhere M is the number of different anatomically defined connection typesand N_(j) ^(i) is the number of connections per anatomical type 1projecting to neuronal unit i. This pre-synaptic input can in turnproduce a post-synaptic activity due to VI connections.POST_(i) ^(VI)(t)=ω·POST_(i) ^(VI)(t−1)+(1−ω)·A _(i) ^(VI)(t)where ω is a term that regulates persistence of activity from previoustime-steps. We can now define the VD contribution to the inputs of uniti as A_(i)^(VD)(t) = ?(POST_(i)^(VD)(t) ⋅ ??(c_(ij) ⋅ s_(j)(t)))?indicates text missing or illegible when filedwhere where φ(x)=0 for x<

, otherwise φ(x)=x. The post-synaptic activity from VD connections isthenPOST^(VD) _(i)(t)=ω·POST_(i) ^(VD)(t−1)+(1−ω)·A_(i) ^(VD)(t)The total post-synaptic influence on the unit i is thereforePOST_(i)=POST_(i) ^(VI)+POST_(i) ^(VD).

This input influence in turn dictates the firing rate of neuronal uniti. The mean firing rate (s_(i)) can range from 0 (quiescent) to 1(maximal firing), such that,f _(i)(t+1)=φ(tan h(g_(i)·(POST_(i)(t)+ω·s _(i)(t))))where g is a scaling factor.

In brain-based devices some connections can modify their strengthdepending on the experience of the device. These plastic connections cancome in two varieties, those that are dependent on a value signal, andthose that are value-independent. Value dependent connections can learnaccording to:Δ

(t+1)=η·s _(i)(t)·s _(j)(t)·BCM(s _(i)(t))·(V(t)−V(t−1))where η is a fixed learning rate, s_(i)(t) and s_(j)(t) are theactivities of the post and pre-synaptic units respectively and V(t) isthe current mean activity in a value neuronal area, which produces asignal indicating that something about the current environment issalient to the learning task. This is a form of temporal differencelearning. The non-linear transfer function BCM( ) can be based on therule of Bienenstock, Cooper, and Munroe. The function BCM can beimplemented as a piecewise linear function, taking post-synapticactivity as input, which is defined by a sliding threshold, θ, twoinclinations (k₁, k₂) and a saturation parameter ρ(ρ=6 throughout):${{BCM}(s)} = \left\{ \begin{matrix}{{{- k_{1}}s};} & {s \leq {\theta/2}} \\{{k_{1}\left( {s - \theta} \right)};} & {{\theta/2} < s \leq \theta} \\{{k_{2}{{\tanh\left( {\rho\left( {s - \theta} \right)} \right)}/\rho}};} & {otherwise}\end{matrix} \right.$

Value independent connections can be modified according toΔ

(t+1)=η·s _(i)(t)·s _(j)(t)·BCM(s _(i)(t))and then can be normalized to prevent unbounded potentiation over time$c_{ij} = \frac{c_{ij}}{\sqrt{\sum\limits_{k = 1}^{K}\text{?}}}$?indicates text missing or illegible when filedwhere K is the total number of connections to unit i.

Examples of devices with hybrid control described below include theSegway Soccer Brain Based Device (SS-BBD) and Darwin++. The descriptionscontained herein indicate how information flows between neural andnon-neural components.

FIG. 2A shows an example of a hybrid device 200. In this example, thenon-neural controller portion 202 includes artificial intelligence to dolocalization. In one embodiment, a localization unit 204 uses data fromand provides data to the neuronal controller portion 206.

The localization unit 204 can be a probabilistic localization unit. Theprobabilistic localization unit 204 can create estimate(s) of theposition of the device. The probabilistic localization unit 204 canrecalculate the estimate(s) with sensor data and a predicted change inposition and orientation caused by actuators. The probabilisticlocalization unit 204 can use Bayesian estimates logic to produce theestimate(s).

In one embodiment, the probabilistic localization unit can use particlefilter localization. Particle filter localization can maintain a numberof potential positions (or “particles”) which can be updated with sensordata and predicted change in position information. Based on the sensordata a probability can be associated with each position. The particlescan be used to produce a final position estimate. In one embodiment, aweighted means of the particles can be used to create the final positionestimate.

In one embodiment, the neuronal controller portion 206 includes aneuronal hippocampus simulation 208 and a neuronal actuator control 210.The localization unit 204 can allow the neuronal controller portion 206to produce a better location determination. The hippocampus simulation208 can produce information to help update the position estimate(s) andprobability estimates for the localization unit 204. In the example ofFIG. 2A, the location determination of the neuronal hippocampussimulation 208 can be sent to the neuronal actuator control 210 whichcan control actuators 212. The laser sensor 214 and odometry sensors 216can be used by both the neuronal controller portion 206 and non-neuronalcontroller portion 202.

The localization unit 204 can be a simultaneous localization and mapping(SLAM) unit. SLAM can simultaneously create maps while localizing therobot's position.

One embodiment of the present invention is a device 200 comprisingsensors 214 and 216, and a controller 220 receiving inputs from thesensors. The controller 220 can include a neuronal controller portion206 and a non-neuronal controller portion 202. The non-neuronalcontroller portion 202 can include a localization module. Thelocalization module 204 can provide data to the neuronal controllerportion. Acutators 212 can receive control commands from the controller220.

The neuronal controller portion 206 can provide data to the localizationunit 204. At least one of the actuators 212 can be controlled by theneuronal controller portion 206. The device 200 can be a robot.

Darwin++ is a hybrid model that combines probabilistic localizationalgorithms with the ability of the brain region called the hippocampusto encode information about location. Darwin++ is based on thehippocampus model of Darwin X (see Appendix), but adds non-neuralcomponents that map the environment and determine the robot's locationon that map. Information flows from the localization module to theneural simulation, and then in turn back from the simulation to alterthe localization module's hypothesis of location. The location of thedevice can be determined through a particle filter, which maintains manylocalization hypotheses (particles). At every time step all theparticles can be updated according to the sensory data and motorcommands, producing a new most likely location for the device.

When a particular particle becomes very unlikely to be the correctlocation given the data, it can vanish from the filter. In a normalparticle filter the hypothesis is replaced with a randomly sampled copyof one of the surviving particles. In one embodiment, it can be replacedwith a hypothesis extracted from the place cell activity CA1. Thissystem is otherwise substantially similar to one in the Appendix, whichwill give further information on the nature of the hippocampal model,the action selection and value dependent learning.

In one embodiment, a hippocampus localization BBD is a navigation systemthat demonstrates localization, path planning, and goal-directedbehavior in natural environments. The system can combine visual, laser,and self-movement information to localize its position and an actionselection system to plan paths. In one embodiment, because the systemrelies on associative memory, the system can be robust to sensor failureand can still navigate effectively when one or more of the sensors arecompromised.

FIG. 2B shows details of the Neural architecture of Darwin++. Sensoryinputs feed through cortical areas before reaching the hippocampalcomplex. Laser information can take two paths to the hippocampus; firstrange data can be directly fed to the hippocampus as sensoryinformation, secondly the non-neural localization algorithms can producean estimate of location on a map of the room that is encodedtopographically in a neural area feeding the hippocampus. Thehippocampal loops can then process the information, with output fromhippocampal area CA1 going to the motor areas, which in turn drive thewheels. The output from CA1 can also be used to produce new locationhypotheses for the localization module. In one embodiment, the neuralsimulation can have 9000 cells and 1.5 million synapses. TABLE 1 Valuesof parameters defining properties of neuronal units in Darwin++. NeuralArea Name Area Size σ-fire σ-vdep ω g Primary Visual Cortex V1 (8) 60 ×80 — — — — Head Direction System HD  1 × 360 — — — — Positive Value R+ 1× 1 — — — — Negative Value R− 1 × 1 — — — — Basal Forebrain BF 1 × 1 — —— — Visual Cortex - Color V2/4-color (4) 6 × 8 0.20 0.10 0.0 1.0 VisualCortex - Spatial tuning V2/4-width (4) 15 × 20 0.20 0.10 0.0 1.0Infemtemporal cortex IT 30 × 30 0.20 0.10 0.0 1.0 IT inhibitoryinterneurons ITi 15 × 15 0.20 0.10 0.15 1.0 Parient cortex PR 30 × 300.20 0.10 0.0 1.0 Anterior Thalamic Nucleus ATN 30 × 30 0.10 0.10 0.501.0 ATN inhibitory interneurons ATNi 30 × 30 0.10 0.10 0.15 1.0 Motorarea for heading M

 1 × 60 0.0 0.10 0.0 1.0 M

inhibitory interneurons M

 1 × 60 0.0 0.10 0.0 1.0 Value System - Positive S+ 4 × 4 0.0 0.10 0.01.0 Value System - Negative S− 4 × 4 0.0 0.10 0.0 1.0 EnterbinalCortex - Input EC_(IN) 30 × 30 0.10 0.10 0.50 1.0 EC_(IN) inhibitoryinterneurons EC_(IN)i 15 × 15 0.02 0.10 0.0 1.0 Enterbinal Cortex -Output EC_(OUT) 30 × 30 0.10 0.10 0.50 1.0 EC_(OUT) inhibitoryinterneurons EC_(OUT)i 15 × 15 0.02 0.10 0.0 1.0 Deniate Gyrus DG 30 ×30 0.10 0.10 0.50 0.75 DG Feedback inhibitory interneurons DG_(ps)i 15 ×15 0.02 0.10 0.0 1.0 CA3 Hippocampal Subfield CA3 15 × 15 0.05 0.10 0.500.75 CA3 Feedback inhibitory interneurons CA3_(pp)i 8 × 8 0.02 0.10 0.01.0 CA3 Feedforward inhibitory interneurons CA3_(pp)i 15 × 15 0.02 0.100.0 1.0 CA1 Hippocampal Subfield CA1 20 × 20 0.05 0.10 0.50 0.75 CA1Feedback inhibitory interneurons CA1_(pp)i 10 × 10 0.02 0.10 0.0 1.0 CA1Feedforward inhibitory interneurons CA1_(pp)i 10 × 10 0.02 0.10 0.0 1.0Estimated position from localization LOC_(IN) 30 × 30 0.2 0.0 0.0 1.0Estimate position from hippocampus LOC_(OUT) 30 × 30 0.02 0.1 0.5 1.0

Table 1 shows values of parameters defining properties of neuronal unitsin one embodiment of a Darwin++ system. Areas V1, HD, LOC_(IN), R+, andR− are input areas and their activity is based on the camera image,odometery, probabilistic localization algorithm, and IR sensorsrespectively, Areas V1 and V2/V4 have 4 sub-areas tuned for color (red,green, blue, and yellow) and 4 sub-areas tuned for varying widthvertical bars. An area with a lower case i in its name (e.g. EC_(IN)i)contains inhibitory neuronal units. The table indicates the number ofneuronal units in each area or sub-area (Size). Neuronal units in eacharea have a specific firing threshold (σ-fire), a threshold above whichvoltage-dependent connections can have an effect (σ-vdep), a persistenceparameter (ω) and a scaling factor (g). Projection Area p cij(0) type φη k1 k2 V1-color→V2/4-color [ ] 1 × 1 0.00 0.03, 0.05 VI — — — —V2/4-color→V2/4-color

[ ] 2 × 2 0.40 0.5, 0.6 VD — — — — V2/4-color→V2/4-color

[ ] 3 × 3 1.00 −0.0012, ‘0.28   VI — — — — V1-width→V2/4-

[ ]1 × 1, 2 × 2, 3 × 3, 4 × 4 1.00 0.008, 0.009 VI — — — —V2-width→V2/4-

[ ] 2 × 2 0.40 0.5, 0.6 VD — — — — V3-width→V2/4-

[ ] 3 × 3 1.00 −0.012, −0.014 VI — — — — V2/V4→IT non-topo 0.03 0.03,0.04 VI — — — — IT→IT [ ] 1 × 1 1.00 0.006, 0.14  VI — — — — IT→FE Θ 2,3 1.00 0.05, 0.06 VI — — — — FE→IT [ ] 3 × 3 1.00 0.36, 0.50 VI — — — —V2/V4→Pr [ ] 3 × 1 0.25 0.25, 0.30 VD — — — — Pr→Pr Θ 4, 6 1.00 0.06,0.08 VI — — — — HD→ATN [ ] 10 × 2 0.20 0.01, 0.02 VI — — — — ATN→ATN Θ10, 15 0.25 0.01, 0.02 VI — — — — ATN→ATN [ ] 1 × 1 1.00 −0.30, 0.50  VI— — — — HD→Maxi [ ] 1 × 1 1.00 0.01, 0.01 VI — — — — Maxi→Maxi Θ 20, 300.50 0.10, 0.20 VI — — — — Maxi→Maxi [ ] 1 × 1 1.00 0.36, 0.50 VI — — —— ATN→EC_(IN) non-topo 0.001 0.40, 0.50 VI — — — — IT→EC_(IN) non-topo0.001 0.40, 050 VI — — — — Pr→EC_(IN) non-topo 0.001 0.40, 0.50 VI — — —— LDC_(IN)→EC_(IN) non-topo 0.001 0.40, 0.50 VI — — — — EC_(IN)→EC_(OUT)non-topo 0.05 0.04, 0.08 VI — — — — EC_(IN)→EC_(IN) Θ 2, 3 0.10 0.05,0.00 VI — — — — EC_(IN)→EC_(IN) [ ] 1 × 1 1.00 −0.60, −1.20 VI — — — —EC_(IN)→DG [ ] 3 × 3 0.10 0.45, 0.60 VI 0.75 0.05 0.00 0.45 EC_(IN)→CA3[ ] 3 × 3 0.05 0.15, 0.20 VI 0.75 0.05 0.00 0.45 EC_(IN)→CA1 [ ] 3 × 30.04 0.30, 0.40 VI 0.75 0.05 0.00 0.45 EC_(OUT)→ATN non-topo 0.01 0.40,0.45 VD — — — — EC_(OUT)→IT non-topo 0.01 0.40, 0.45 VD — — — —EC_(OUT)→Pr non-topo 0.01 0.40, 0.45 VD — — — — EC_(OUT)→EC_(IN)non-topo 0.05 0.04, 0.08 VI — — — — EC_(OUT)→EC_(OUT) Θ 2, 3 0.10 0.45,0.60 VI — — — — EC_(OUT)→EC_(OUT) [ ] 1 × 1 1.00 0.90, 1.20 VI — — — —DG→CA3 [ ] 3 × 3 0.00 0.45, 0.60 VI — — — — DG→DG [ ] 3 × 1 1.00 0.30,0.50 VI — — — — DG→DG_(FB)i Θ 2, 3 0.10 0.45, 0.60 VI — — — —DG_(FB)i→DG [ ] 1 × 1 1.00 −0.90, −1.20 VI — — — — DG→CA3_(PP)i Θ 2, 30.10 0.45, 0.60 VI — — — — CA3_(PP)i→CA3 [ ] 1 × 1 1.00 −0.90, −1.20 VI— — — — CA3→CA1 [ ] 3 × 3 0.08 0.45, 0.60 VI 0.75 0.05 0.90 0.45 CA3→CA3non-topo 0.10 0.15, 0.20 VI 0.75 0.05 0.90 0.45 CA3→CA3_(FB)i Θ 2, 30.10 0.45, 0.60 VI — — — — CA3_(FP)i→CA3 [ ] 1 × 1 1.00 −0.90, −1.20 VI— — — — CA3→CA1_(FP)i Θ 2, 3 0.10 0.45, 0.60 VI — — — — CA1_(FP)i→CA1 [] 1 × 1 1.00 −0.90, −1.20 VI — — — — CA1→EC_(OUT) [ ] 3 × 3 0.25 0.60,0.75 VI 0.75 0.05 0.90 0.45 CA1→M_(ADG)

[ ] 3 × 3 1.00 0.01, 0.02 VD — 0.05 0.90 0.45 CA1→M_(ADG)

[ ] 3 × 3 1.00 0.01, 0.02 VD — 0.05 0.90 0.45 CA1→S+

[ ] 3 × 3 1.00 0.01, 0.02 VD — 0.005 0.90 0.45 CA1→S−

[ ] 3 × 3 1.00 0.01, 0.02 VD — 0.0005 0.90 0.45 CA1→LDC_(OUT) [ ] 1 × 11.00 0.01, 0.02 VD — 0.001 0.90 0.45 R+→S+, R−→S− non-topo 1.00 0.25,0.25 VI — — — — BF→ EC_(IN)EC_(OUT)DG, CA3, CA1 non-topo 0.05 −0.01,−0.02 VI — — — — S+→M_(ADG) non-topo 1.0 0.05, 0.06 VD — — — —S−→M_(ADG) non-topo 1.0 −0.05, −0.06 VD — — — — LDC_(IN)→LDC_(OUT) [ ] 1× 1 1.0 0.3, 0.5 VI — — — —

Table 2 shows properties of anatomical projections and connection typesin an embodiment of a Darwin++ system. A presynaptic neuronal unitconnects to a postsynaptic neuronal unit with a given probability (p)and given projection shape (Arbor). This arborization shape can berectangular

with a height and width (h×w), doughnut shaped or a ring of lateralconnections “Θ” with the size of the ring constrained by an inner andouter radius (r1, r2), or non-topographical “non-topo” where any pairsof presynaptic and postsynaptic neuronal units have a given probabilityof being connected. The initial connection strengths, cij(0), are setrandomly, with a uniform distribution, within the range given by aminimum and maximum value (min, max). A negative value for cij(0),indicates inhibitory connections. Connections marked with “intra” denotethose within a visual sub-area and connections marked with “inter”denote those between visual sub-areas. Projections marked # are positivevalue-dependent; they use the S+ area for TD learning. Projectionsmarked * are negative value-dependent and use the S− area for TDlearning. A connection type can be voltage independent (VI), orvoltage-dependent (VD). denotes the persistence of the connection.Nonzero values for η, k1 and k2 signify plastic connections.

FIG. 3A shows another example of a hybrid device 300. In this example,information from the sensors, such as IR sensor 302, camera 304, andlaser sensor 306 can be sent to non-neural sensor processing 308. Thenon-neural controller 310 can control at least one actuator, such asobject manipulator actuator(s) 312. The neuronal controller portion 314can control at least one actuator, such as the camera orientationactuator 316. In one embodiment, the neural controller portions 314 andnon-neural controller portion 310 can share the control of an actuator,such as the movement actuator 318. For example, the neural controllerportion 314 can control the movement actuator(s) 318 with an override bythe non-neuronal controller portion 310.

The neuronal and non-neuronal control portions 314 and 316 can each havesensor processing portions 320 and 308 and actuator control portions 322and 324. Signals can be sent between these sections as needed.

In one embodiment, laser and IR data can stay completely on thenon-neural controller side and can be used to generate motor commandsand actuate the ball-catchers by non-neural controllers. A pathway canbe positioned from the sensory processing component to the actuatorcomponent.

In one embodiment, the device 300 comprises sensors 302, 304 and 306; acontroller 301 receiving inputs from the sensors 302, 304 and 306; thecontroller 301 including a neuronal controller portion 314 and anon-neuronal controller portion 310; and acutators 316, 318 and 312receiving control commands from the controller portion. At least anotherone of the actuators can be controlled by the non-neuronal controllerportion 310.

In one embodiment, the actuator 318 is controlled by both the neuronalcontroller portion 314 and non-neuronal controller portion 310.

The device 300 can be a robot, such as a mobile robot. The neuronalcontroller portion 314 and the non-neuronal controller portion can be onthe robot.

The Segway Soccer Brain-Based Device (SS-BBD) is an exemplary hybridcontrol system, comprising of both a simulated nervous system neuronalcontrol position and more traditional robot control algorithms. Thesimulated nervous system can be based on a simplified model ofvertebrate nervous system. The neural simulation can be primarilyresponsible for sensorimotor control, and the non-neural controllers canprovide higher level action selection. FIG. 3B depicts an architectureof an exemplary hybrid control system, which can be described asfollows. IR sensors, a camera and a laser rangefinder provide sensoryinput for the system. Action Selection can be the central module; it canreceive non-neural input from the IR sensors about the nearby objectsand also gets visual data interpreted by the neural visual system.Action Selection can send direct commands to some actuators and indirectones to the SS-BBD's wheel through Head and Body movement neuronalareas. The device's movement may also be affected by an obstacledetection and avoidance mechanism (Route Planning System).

The early stages of vision can be of a non-neural nature. RGB cameraimages can be converted by mathematical algorithms to YUV colorspaceimages. The Y component (light intensity information) can beedge-detected by mathematical algorithms. The UV components (colorinformation) of the image can be compared previously generated lookuptables of salient colors. The lookup tables can represent theprobability that a specific color in the image matches one of thesalient colors.

Neural processing can begin in the next stage; edge-filtered images canbe turned into activity values in neuronal groups that represent edgeinformation and the probabilistic representations of salient colors canbe turned into activity in neuronal groups representing these colors.These groups can be roughly equivalent to areas in the primary andsecondary visual cortices.

These color and edge groups can be connected to neuronal groups whoseactivity represents the presence of certain salient objects in thevisual field, such as soccer balls. These groups can have a functionthat is similar to that of inferotemporal and parietal cortices.

Motor neuronal areas can control the head (pan-tilt camera) and body(wheel speeds) of the SS-BBD. Activity in these groups can drive thedevice to approach whatever target object is currently being tracked,unless there is an overriding motor.

The ball-handling actuators can driven directly by the non-neural actionselection controllers.

A non-neural algorithm can be responsible for deciding what object issalient at any given time, and whether that object should be approachedor merely tracked while staying in place. For instance, the SS-BBD mightbe tracking an opponent but then the ball rolls nearby and should becaptured. The action selection mechanism can be responsible for makingthe decision to change targets. Action selection can use informationfrom the neural visual areas representing target objects to make thesedecisions. It also can use non-neurally processed information from IRsensors. Action selection can create a temporary connection betweenwhatever object neuronal group it decides to track (e.g., Ball) and theHead neuronal group, therefore creating the neural control loop to trackthat object automatically.

A non-neural algorithm can use information from the laser rangefinderand the IR sensors to determine if there is an obstacle nearby thatneeds to be avoided. In that case it tries to balance the need toapproach the current target object (selected by the action selectionmechanism) with the need to not strike the obstacle. The informationabout obstacle location can come from the laser data, and theinformation about obstacle location can be extracted from activities inthe target object neuronal area. When an obstacle is present, motoroutput from the obstacle avoidance module can override output from theBody neuronal group.

FIG. 4 shows a SS-BBD device. These devices can include: (A) activecapture devices, (B) laser rangefinder, (C) pan-tilt unit and camera,(D) kicking assembly, (E) passive capture ring, (G) crash bars.

The SS-BBD can be based on the Segway Robotic Mobility Platform, acommercially available robot version of the Segway Human Transporter. Analuminum chassis sits on the commercial base, containing a cluster ofsix compact Pentium IV PCs and enough battery capacity to power it for45 minutes. A set of shock-mounted crash arms protect the device in caseof collision or failure of the balance controller. The SS-BBD canpossess various sensory systems, including a color camera, a laserrangefinder, and a digital compass. Banks of short range IR proximitysensors can be mounted low around the device to detect nearby soccerballs. One bank of IR sensors can be mounted on the back above ballheight to detect nearby non-ball obstacles that are outside the sensingarc of the front-mounted camera and laser rangefinder.

The SS-BBD can possess solenoid actuated devices that enable it tohandle a soccer ball. A pair of flexible jaw-like plastic catchers canpinch a ball firmly against the front of the device, allowing it torotate in place with a ball. To kick the ball, the catchers can beraised and a second set of solenoids can actuate a four-bar linkage thatdelivers thrust to the ball. A passive device aids in capturing incomingballs on the sides and back: a ring of flexible plastic mounted onstraps can absorb the impact of incoming balls, allows them slip underthe ring, and then traps them against the device. The low mounted IRsensors can then detect on which side of the device the ball is located,and a simple pivot-in place motion allows the device to turn until theball is on its front face, at which point it captures the ball with thesolenoid-driven catchers.

FIG. 3B describes the overall architecture of a control system of oneembodiment, which can be described as follows. IR sensors, a firewirecamera and a laser rangefinder can provide sensory input for the system.Action Selection can have the central module; it can receive input fromthe IR sensors about the nearby objects and also get visual datainterpreted by the neural visual system. Action Selection can senddirect commands to some actuators and indirect ones to the SS-BBD'swheel through Head and Body movement neuronal areas. The device'smovement may also be affected by an obstacle detection and avoidancemechanism (Route Planning System).

In one embodiment, to ensure effective behavior, robot soccer canrequire fast, robust visual information processing to identify objectson the playing field. In this section, we describe the methods andarchitecture we applied to satisfy this requirement.

The visual information can be provided to the SS-BBD by a Sony IEEE 1394camera equipped with a wide angle lens working at 30 frames per secondand 640×480 pixels. The raw sensory pixel data can be separated intoluminance and color channels (YUV colorspace). FIG. 3C can show how thisvisual information is processed by the neural simulation; the luminanceinformation can feed a set of edge detecting neuronal areas and thecolor information can drive neuronal areas dedicated to detectingparticular game-related colors. Information from these early visualareas is combined in neuronal areas that detect game-related objects.

In one embodiment, there are five important objects on a robot sensorfield with different colors or color combinations: our goal, opponent'sgoal, ball, teammate and opponent. The visual system can use sixneuronal color groups, each of them having a preferred color, namelyRed, Green, Blue, Yellow, Pink and Purple (see FIG. 3C). To speed up themainly color-based object recognition, a tool can recognize thesepreferred colors by creating a lookup table for each color on the UVcolor space. A value in a color table can be regarded as the probabilityof a particular UV coordinate belonging to that specific color.Snapshots from an exemplary user interface to create a color lookuptable is presented in FIG. 5.

The visual and object recognition nervous system can contain 15 neuronalareas. FIG. 3C shows a high level diagram of the system including thevarious neuronal areas and the arrangement of the feed forward andrecurrent excitatory and inhibitory synaptic connections. The model isbased on recent model of the primate visual system, where the edge andcolor filtering neuronal areas correspond roughly to primary visualcortices and the object recognition neuronal areas perform similarfunctions to inferotemporal and parietal cortices. The color and edgegroups of the neuronal visual areas can be direct and topographicconnection with the preprocessed visual sensory inputs (FIG. 3C):

-   -   Each of the six color neuronal groups can have activity        proportional to how close an image matches its preferred color        as determined by its color lookup table. That is, activity of        each cell in a color group denotes the ‘closeness’ of it's pixel        in the image to the preferred color. For example, for a picture        with a red ball on a green grass, the activities of the red        group can be almost everywhere approximately zero (silent)        except on coordinates where the ball is.    -   A vertically, a horizontally, and two diagonally edge-filtered Y        channel can be directly mapped on to the corresponding four edge        groups. In the SS-BBD's neural model, each object the robot had        to recognize on the playing field can have its own neuronal        area, namely Ball, Goal, Teammate, Opponent, Opponent's Goal        areas (See FIG. 3C). These object groups can receive their        topographical inputs from one or more color neuronal groups.

Some objects can be detected using a single color. For example, the ballcan be red, the goal can be purple, and the opponent's goal can be pink.Teams, however, can be denoted by combinations of colors: for example inone case, our team is purple and yellow on the same color marker. Forthese objects, the corresponding neuronal area can recognizeconjunctions of these colors in the nearby topographic locations. Theball neuronal area can also recognize the ball by its shape: aconjunction of edges in a particular configuration. The object areas canhave recurrent self-excitatory connections and some also had inhibitoryconnections to the other object areas. Inhibitory connections made thesystem can be more robust: Object groups that should not be in the sameplace in the visual field can have inhibitory connections to each other.To put it all together, the visual object recognition can be based oncolor information, tuned self-excitation, and cross inhibition betweenthe object groups and shape information. All these together made itpossible for the neural simulation to efficiently recognize objects.

FIG. 3C illustrates a schematic neural architecture for the visualsystem and object detection, which has the following features: Schematicneural architecture for the visual system and object detection. 1) RGBto YUV conversion of the grabbed image. 2) Edge filtering of the Y(luminance) channel and Color Filtering in the UV domain. 3) The edgeand color neuronal areas are connected with the object neuronal groups.4) Cross/lateral excitations and recursive inhibitions help improve theobject recognition. All the connections in the processing stream can betopographical. Lines with filled circles can mean inhibition, while anarrow can denote excitatory connection.

FIG. 5 shows creating a color specific Gaussian envelope lookup table inthe UV domain. The user can crop out an area of the visual field so thatthe cropped area consists of only the selected color (object), which isa red ball on the above example. A statistical algorithm then reads theRGB pixel values of the cropped area, convert them to YUV and, basedonly on the UV values, generate a Gaussian like two dimensionalprobability surface on the UV space. This surface can be saved and usedas a lookup table for that specific color.

When humans look for an object, both head and eyes turn in the properdirection, but the eyes arrive first due to their faster movement. Afterthe eyes have completed the saccade to the object, they enter a smoothpursuit phase and track the object as the head completes its turn, andthe eyes are once more centered in the head. When the body is alsoinvolved in visual search, as in the case of traveling to a target, thenthe eye-head coordination is followed by the body re-centering the head.

The target tacking behavior the SS-BBD can be analogous to humanbehavior: a camera saccade can foveate the target object and then thebody turns to follow while the camera tracks the object smoothly. Thefollowing paragraphs describe the neural mechanisms that can implementthis algorithm. The activity of neuronal object areas can be organizedretinotopically, and whichever one of these areas is selected to be thetarget object is projected retinotopically to a pan area and a tiltarea. The pan area's activity therefore can reflect the horizontaldistribution of activity in the target area. Activity in the pan areacan drive the panning actuator in a topological fashion: more activityto the right of the area means the camera turns to the right. Thus thecamera can turn to the right to re-center an object that is in the righthalf of the image. The tilt area can work similarly in the verticaldirection. Information about the velocity of the target object can alsomodify the activities of the pan and tilt areas. This can allow the gazesystem to track objects that are moving faster than would be possible ifonly positional information was used. The apparent velocity of thetarget object can be continuously calculated. Two slip areas (one forvertical position, one for horizontal) can have activities thatrepresent where the target object is anticipated to be, given itscurrent position and velocity. The horizontal slip area can project tothe pan area, and the vertical slip area can project to the tilt area.The result of this architecture can be that the camera will tend to leada moving object rather than lag it, and the body motion that follows thehead will therefore have a smoother trajectory.

The activity of the pan area can, in turn, project topographically tothe body motor area, which controls the rotational velocity of theSS-BBD. Off center activity in the body area can create a turn in thatdirection. The total system can result in a cascade of activity thatcomes from visual areas to the motor areas, which in turn creates achange in behavior and changes in the visual input. The smooth motion oftracking behavior can be somewhat surprising given that head-bodytracking is underconstrained. But the very different time constants ofcamera and body motion prevent tracking behavior from being underdampedin most circumstances.

The non-neural elements of the controller can be organized on twolevels: behaviors and plays. Behaviors can be atomic actions that thedevice undertakes, such as searching for an object, or kicking the ball.Each behavior can be a separate controller that sends commands to themotor neuronal areas independently of the other behaviors. In oneembodiment, only one behavior can be active at any one time and can takeinput directly from the sensors or input pre-processed from the sensoryneuronal areas. A basic play can be composed of a sequence of behaviors,and the sensory conditions that will cause the controller to transitionto each new behavior. Plays can also be compositional; a morecomplicated play can be composed of other plays as well as behaviors.All plays can draw on the same set of possible behaviors, but recombinethem in various way to produce different results. Finally, a mechanismcan exist that allows the human player to over-ride the normalautonomous execution of plays by the SS-BBD. A voice command system canbe installed on the human player's Segway Scooter that wirelessly relaysa command to the SS-BBD causing it to execute a new play requested bythe human player.

The behaviors used by the SS-BBD can be: Find Object: Pan the camera andturn the SS-BBD in place, until a target object is visually recognized.Track Object: Approach a target object while keeping it foveated usingboth the pan-tilt and the wheels of the SS-BBD. Capture Ball: Performthe final maneuvering to get close to the ball and lowers the catchersto trap it. Kick: Lift the catchers away from the ball and fire thekicking solenoids. The Kick and Capture Ball behaviors can be simple,precisely tuned controllers. Capture Ball can be a proportionalcontroller that uses feedback from both the neuronal visual ball areaand non-neural signals direct from the IR to guide the SS-BBD to thecorrect point to capture the ball, at which point is actuates thesolenoids. Kick can be an open-loop sequence of commands to catchersolenoids, kicker solenoids, and the wheels that results in a straight,powerful kick traveling about 1.5 m/s. Both behaviors can have someerror correction logic that can correct or re-start the process if theball is not successfully kicked or captured.

The behaviors Find Object and Track Object can be set to work on anyobject that is detected by the neural simulation: soccer balls, eithergoal on the field, a teammate, or an opponent. Find Object can have twodifferent algorithms depending on the circumstances of its execution:(1) if called during a momentary loss of an object that had beenpreviously tracked, (2) if called for a brand new search. The first casecan occur if there is any velocity information about the target object,then Find Object will use it to steer the search in the same directionin which the object was observed to travel. This velocity signal canallow Find Object to continue to follow the path of an object that hasbeen momentarily occluded by some obstacle. Otherwise, Find Object cansearch in a random direction for the target object. Since the pan-tiltunit can track the camera faster than the device's body can turn, itfirst performs a head-check to the limits of the pan for the object. Ifnot found, it can smoothly transition the head motion into a body motionin the same direction.

The Track Object behavior can be based on psychophysical data abouteye/head-body coordination, and can be implemented using neuralmechanisms. Track Object can cause the SS-BBD to move forward as well asto turn towards the target when it is not in possession of the ball(e.g., while chasing a loose ball). If it is in possession (e.g., whenlining up a shot on the goal) then it can just rotate in place. Whenmoving forward Track Object uses a planned obstacle avoidance algorithmthat balances target seeking with obstacle-avoiding. When humans arefaced with the task of trying to reach a desired location while avoidingobstacles along the way, their navigation behavior can be described andpredicted by a simple dynamical model. The SS-BBD can use this humanpsychophysics model to plan routes when executing the behavior TrackObject in the presence of obstacles.

The model can control angular velocity of the SS-BBD as a function ofthe angular heading and distance to both the target and the obstacles,where the existence of multiple obstacles can be handled by linearlycombining terms of each obstacle. The heading of the agent (φ) obstacledistance (d_(o)), and heading (ψ_(o)), and target distance (d_(t)) andheading (ψ

) are the variables in a spring-mass type equation to determine angularvelocity of the agent ? = −1/3(ϕ − ?) − ?1980(ϕ − ?)???indicates text missing or illegible when filedwhere the first term is a spring function governing attraction to thetarget object, and the second term is a function dictating repulsionfrom obstacles. In this equation, headings are in units of radians anddistances in units of meters. There are differences between the modelused by the SS-BBD and the original model meant to replicate humanbehavior. The human model can possess a damping term (i.e., a functionof φ) whereas this exemplary embodiment does not. The attraction to thetarget object need not be related to the distance to the target in thismodel, whereas the human model can bean exponential distance term. Thismodel responds to obstacles that are further away from the robotheading, and more strongly to those obstacles at the same deflection.The changes to the obstacle term and the removal of the damping term canbe used since the SS-BBD has substantially different dynamics than ahuman being. The lack of distance dependency in the target object termcan be used because we use visual information to find the target object,which easily provides good heading information but poor distanceinformation without substantial image processing.

In contrast, obstacles can be detected through use of a laserrangefinder, which provides accurate distance and heading information.An obstacle can be defined as any set of adjacent laser readingssubtending at 3° of arc that are all less than two meters from therangefinder. The values ψ_(o) and d_(o) for the obstacle can then beequal to the average values of those three laser readings. The 3°requirement provides a measure of resistance to false positive detectionof obstacles. Since this device operates in a real environment withstringent safety requirements, the translational and rotational velocityof the SS-BBD are subject to additional constraints. The normal maximumforward speed of the device can be 5 m/s, but this can be reducedproportional to the average distance of all obstacles detected by thelaser rangefinder. This can increases safety by reducing speed incluttered environments. Additional safety can be provided by backing upslowly if an obstacle is less than 1 m away in the front arc, as long asthere are no obstacles within 1 m in the back arc (as determined by rearIR sensors).

FIG. 7 shows Finite state automata showing how three basic plays can becomposed of behaviors and conditional transitions between them. WhenPlay Select is reached another play is chosen and executed. Examples oftypical avoidance paths using these algorithms can be seen in FIG. 6.Plays can be a higher level of control than behaviors. Plays may becomposed of behaviors or even other plays. A play can be a finite stateautomation where the state is a particular behavior, action, or otherplay. As in all finite state automata, there are a set of conditionaltransitions between the states that determine the flow of control thatdefines the play Most of the plays used on the SS-BBD can be at thelevel of very simple actions on the field, such as ‘chase the ball’, or‘pass to the teammate’, or ‘run downfield until near the goal’. FIG. 7shows finite state automata that represent the basic plays Chase, Pass,and Run Downfield. In one embodiment, other basic plays in therepertoire can be Shoot (which is identical to pass, except the targetis the opponent's goal), Run Upfield (identical to Run Downfield, exceptfor the target), Follow the Teammate, Mark the Opponent, Block our Goal,and Wait (stay in place, tracking the ball, but not moving forward untilthe ball is near enough to capture). The SS-BBD can also possesses anumber of plays that are more complicated, composed of the basic plays.One such is a simple offensive play designed to move the ball down thefield. The flow of the play goes from Chase to Pass to Run Downfield.When Run Downfield is complete it can return to Chase and the sequencecan repeat. Each sub-play in the sequence can transition to the nextsub-play when it has finished its normal execution.

At any point the flow of control can be interrupted by a voice commandfrom the human teammate. The Segway HT scooter ridden by the human has avoice recognition board such as a (Voice Extreme™ Toolkit, Sensory,Inc., Santa Clarita, Calif.) connected to a microphone that canrecognize about a dozen discrete short phrases after being trained to aparticular voice. These phrases can be code names for various plays inthe SS BBD's repertoire, and for each such phrase a correspondingcommand is transmitted wirelessly to the SSBBD, which upon receiving thecommand immediately executes the correct play.

Although mimicking biology has in recent years become a well-acceptedapproach to building robot system, we are particularly interested inbehavioral control at the level of systems neuroscience. Large-scaleneural simulations of vertebrate neuroanatomy and neurophysiology can beused to test hypotheses of how real nervous systems function. We feel itis important that such neural simulations are embodied in a real-worlddevice interacting with a rich environment, rather than operating inabstract simulation worlds. Much of the complexity of animal behaviorresults from the interactions between the nervous system, the rest ofthe body, and the environment. The Brain-Based Device approach offersinsights for designing robots that need to survive and accomplish tasksin dynamic environments with robustness and adaptability that isdifficult to obtain using traditional engineering and artificialintelligence methods. We applied the BBD method to the design of thisSegway Soccer robot, and produced a device that is capable ofinteracting with humans both cooperatively and competitively, but mostimportantly, safely.

In these two devices there are exemplary operating principles thatdescribe how the neural and non-neural control elements interact witheach other.

The amount of activity in a neuronal group can cause a non-neuralcontroller to make a decision. For example, if summed activity ofneuronal units in the Ball area exceeds a threshold the ball is presentin the field of view, and the action selection unit may decide to chasethe ball.

The location of activity in a topographically mapped neuronal group canused by a non-neural controller as a control signal. For example, whiletracking a ball, an obstacle is found in front of the SS-BBD. The centerof mass of activity in the Ball area can produce an x, y value thatcorresponds to the ball's azimuth and heading in the visual field. Theball's heading can be used by the obstacle avoidance algorithm inconjunction with the obstacle heading to determine in what direction theSS-BBD should drive.

Non-neural controls can create temporary connections between neuronalgroups. For example, the action selection module decides that the ballshould be chased. It creates a connection from the Ball group to theHead motor group, and the resulting cascade of neuronal activity causesthe SS-BBD to chase the ball.

Non-neural controllers can override the motor output of the neuralcontrollers in certain circumstances. For example the obstacle avoidancemodules can inhibit the output of the Head-Body neuronal controllerswhen it detects an obstacle in favor of its own motor commands.

Both neural and non-neural controllers can share the same input stream.For example, Darwin++ has the laser range data going directly into asensory cortical area. The same data can also be processed by thelocalization module to determine location on a map.

A circular flow of information between neural and non-neural controllerscan be established that refines an estimate. For example, in Darwin++the localization module produces an estimate of location. This estimatein turn is one of several sensory modalities that is fed to ahippocampal complex model. Neuronal units in CA1 area of the hippocampushave activity that is strongly tied to place; therefore the activity ofthese units can be used to produce new hypotheses of location for thelocalization model, which closes the circle.

A non-neural controller can modulate neural activity. In one version ofthe SS-BBD, a model was implemented where instead of having a temporaryconnection from the Ball group made to the Head group duringball-chasing, there was connections from all object areas to the Headgroup. However, the action selection module would increase the basalactivity of the area that was of interest, and allow cross-inhibitionbetween object areas cancel out other signals to the Head motor area.

In one embodiment, the code has roughly 20,000 lines, and consists ofthe following major modules: Main: Infrastructure of the software;startup, shutdown, and communication between other modules (1 k lines).Display: User interface and visualization of the current state of theneural simulation (1.5 k lines). Vision: Low-level operation of thecamera and image pre-processing (3 k lines). Sensors: Low-leveloperation of the IR sensors, compass, and laser rangefinder (2.5 klines). Segway: Communication with the Segway Robotic Mobility Platform(800 lines). Actuators: Low-level operation of the pan-tilt unit andsolenoids (700 lines). Voice: Communication with the human player'svoice command module (300 lines). Simulator: The neural simulation (5 klines). Action Selection: Non-neural algorithms that control devicebehavior (3 k lines). Localization: Code that uses probabilisticalgorithms to estimate the location of the device in a global coordinatesystem (1 k lines, only in Darwin++). that is, roughly one quarter ofthe total code base is devoted to neural control, one sixth tonon-neural control, one third to low-level sensor and actuatoroperations, and the remainder to infrastructure, communication, and userinterfaces. TABLE 3 Values of parameters defining properties of neuronalunits Neuronal groups in the Segway Soccer BBD. Description Areas Sizeσ-fire σ-vdep ω g Color Visual Areas (6) Orange, Green, Blue, 60 × 80 —— — — Yellow, Pink, Purple Edge Visual Areas (6) Horiz, Vert, 60 × 80 —— — — LeftDiag, RightDiag Aniticipated Object Location (2) ObjectX,ObjectY  1 × 200 — — — — Pan-tilt Position Sensing (2) PanPos, TiltPos 1 × 200 — — — — Object Areas (5) Ball, 20 × 20 0.05 0.10 0.25 1.0OurGoal, TheirGoal, Teammate, Opponent Motor Areas (5) BodyTurn,BodySpeed,  1 × 200 0.10 0.10 0.0 1.0 PanMotor, TiltMotor

Table 3 shows values of parameters defining properties of neuronal unitsin one Segway Soccer BBD. The visual, anticipated object location, andpan-tilt position areas are input areas, and their activity are createdby non-neural components of the hybrid (by respectively, visualprocessing algorithms, mathematical transformations of object areaactivity, and commands sent to the pantilt unit). Object areas representwhere in the visual field certain items are located. Activity in motorareas is used to generate commands to various actuators. The tableindicates the number of neuronal units in each area or sub-area (Size).Neuronal units in each area have a specific threshold (σ-fire), athreshold above which voltage-dependent connections can have an effect(σ-vdep), a persistence parameter (ω) and a scaling factor (g). TABLE 4Properties of anatomical projections and connection types in the SS-BBD.Connections between neuronal groups in the Segway Soccer BBD. ProjectionArbor p cij(0) type Orange→Bail [ ] 3 × 1 1.00 0.8, 1.0 VI Berl, Harte,LeftDrag, M 1 × 1 1.00 1.3, 1.5 VI RightDrag→Bail Bail→Bail Θ 3 × 201.00 −5.0, 5.5   VI Bail→Terminate [ ] 1 × 1 1.00 −5.5, 5.6   VIBail→Opponent [ ] 3 × 1 0.40 −5.5, 5.6   VI Bail→Bail [ ] 3 × 1 1.000.6, 0.8 VD Purple→OurGoal [ ] 3 × 1 1.00 0.8, 1.0 VI OurGoal→OurGoal Θ10 × 20 0.75 −5.0, −5.5 VI OurGoal→Terminate [ ] 1 × 1 1.00 −5.5, 5.6  VI OurGoal→Opponent [ ] 1 × 1 1.00 −5.5, 5.6   VI OurGoal→OurGoal [ ] 1× 1 0.40 0.6, 0.8 VD Purple, Yellow→Terminate M 1 × 1 1.00 0.8, 1.0 VITerminate→Terminate Θ 4 × 20 0.75 −5.0, −5.5 VI Terminate→Bail [ ] 1 × 11.00 −5.5, 5.6   VI Terminate→OurGoal [ ] 1 × 1 1.00 −5.5, 5.6   VITerminate→TheirGoal [ ] 1 × 1 1.00 −5.5, 5.6   VI Terminate→Terminate [] 3 × 1 0.40 0.6, 0.8 VD Pink, Blue→Opponent M 1 × 1 1.00 0.8, 1.0 VIOpponent→Opponent Θ 4 × 20 0.75 −5.0, 5.5   VI Opponent→Bail [ ] 1 × 11.00 −5.5, 5.6   VI Opponent→OurGoal [ ] 3 × 1 1.00 −5.5, 5.6   VIOpponent→TheirGoal [ ] 3 × 1 1.00 −5.5, 5.6   VI Terminate→Terminate [ ]3 × 1 0.40 0.6, 0.8 VD Pink→TheirGoal [ ] 3 × 1 1.00 0.8, 1.0 VITheirGoal→TheirGoal Θ 10 × 20 0.75 −5.0, 5.5   VI TheirGoal→Terminate [] 3 × 1 1.00 −5.5, 5.6   VI TheirGoal→Opponent [ ] 3 × 1 1.00 −5.5,5.6   VI TheirGoal→TheirGoal [ ] 3 × 1 0.40 0.6, 0.8 VDObject_X→PanMotor [ ] 3 × 1 1.00 0.30, 0.40 VI Object_Y→TiltMotor [ ] 3× 1 1.00 0.30, 0.40 VI PanPos→BodyTurn [ ] 3 × 1 0.25 0.03, 0.08 VITiltPos→BodySpeed [ ] 3 × 1 0.25 0.03, 0.04 VI

Table 4 shows an example of properties of anatomical projections andconnection types in a SS-BBD of one embodiment. A presynaptic neuronalunit connects to a postsynaptic neuronal unit with a given probability(p) and given projection shape (Arbor). This arborization shape can berectangular

with a height and width (h×w), doughnut shaped or a ring of lateralconnections “Θ” with the size of the ring constrained by an inner andouter radius (r1, r2), or non-topographical “non-topo” where any pairsof presynaptic and postsynaptic neuronal units have a given probabilityof being connected. The matching arborization “M” denotes a logical ANDtopographical projection from all pre-synaptic groups to thepost-synaptic group. The initial connection strengths, cij(0), are setrandomly, with a uniform distribution, within the range given by aminimum and maximum value (min, max). A negative value for cij(0),indicates inhibitory connections. Connections marked with “intra” denotethose within a visual sub-area and connections marked with “inter”denote those between visual sub-areas. Projections marked # are positivevalue-dependent; they use the S+ area for TD learning. Projectionsmarked * are negative value-dependent and use the S− area for TDlearning. A connection type can be voltage-independent (VI), orvoltage-dependent (VD). φdenotes the persistence of the connection.Non-zero values for η, k1 and k2 signify plastic connections.

Functions of the hybrid control device, such as those represented in theFigures and Table above can be implemented using software (also calledcontrol logic) executing in computer(s) each having one or moreprocessors. The software, when executed, causes the processors toperform the functions described herein. In embodiments, such as softwareis stored in computer useable mediums (such as, but not limited to,magnetic or optical storage mediums, carrier waves, etc.) of computerprogram products (such as, but not limited to, RAM, ROM, CDs, DVDs, harddrives, memory stocks, electrical signals over the Internet, etc.).

The foregoing description of preferred embodiments of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Many modifications andvariations will be apparent to one of the ordinary skill in the relevantarts. The embodiments were chosen and described in order to best explainthe principles of the invention and its partial application, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with various modifications that are suited tothe particular use contemplated. For example, the features and functionsdescribed herein are applicable to other applications in addition to ahybrid device capable of playing soccer. It is intended that the scopesof the invention are defined by the claims and their equivalents.

APPENDIX

A brain-based device (BBD) can be able to adapt to varying terrain bylearning which aspects of the terrain it can traverse and which aspectsit must circumvent. The BBD can make decisions on aspects of theenvironment that go beyond mapmaking of prior robotic systems. Thesedecisions can require perceptual categorization of local objectfeatures, which the BBD of the present invention makes, thereby enablingthe BBD to plot a course.

The BBD of the present invention can include a neuronal control portionincluding a simulated nervous system having neural areas, in oneembodiment, the hippocampus, the parietal cortex, the inferotemporalcortex, and a thalamic head direction system, The bi-directionalconnectivity between the hippocampus and these regions (see Diagram 1A),as well as the intricate connectivity within the hippocampus (seeDiagram 1B), can comprise a system that loops multimodal informationover time and allows the formation of associative memories with atemporal context. The BBD also has a mobility base enabling it to moveabout in a real world environment under control of the simulated nervoussystem, together with a multi-sensor input enabling the BBD to recognizeobjects and select an action that has value consequences.

Diagrams 1A and 1B show examples of a neuronal control portion that canbe modified by adding connections to a non-neuronal control portion.

Diagram 1A is a schematic of the regional and functional neuroanatomy ofa neuronal controller portion 12 of one embodiment, which includes theintricate anatomy and physiology of the hippocampus and its surroundingregions This neuronal controller portion 12 can integrate spatialsignals from numerous objects in time and provide flexible navigationsolutions to aid in the exploration of unknown environments, such as theenvironment 28. By simultaneously sampling from its neural regionsduring a navigation task, the architecture of the hippocampus, asdescribed more fully below, provides a means to organize multi-modalinformation over different timescales which is important for thedevelopment of spatial and episodic memories.

The neuronal controller portion 12, as shown in Diagram 1A, is modeledon the anatomy and physiology of the mammalian nervous system but, ascan be appreciated, with far fewer neurons and a much less complexarchitecture. The neuronal control portion 12 can includes a number ofneural areas labeled according to the analogous cortical and subcorticalregions of the human brain. Thus, Diagram 1A shows respective neuralareas labeled as V1 Color, V2/4 Color, IT, V1 Width, V2,4 Width, Pr, HD,ATN, BF, Hippocampus, R+, S, R−, and M_(HDG). Each neural area V1 Color,V2/4 Color, etc. can contain different types of neuronal units, each ofwhich can represent a local population of neurons. Each ellipse shown inDiagram 1A denotes a different neural area, with each such area havingmany neuronal units. To distinguish modeled or simulated neural areasfrom corresponding regions in the mammalian nervous system, thesimulated areas are indicated in italics, e.g. IT.

The neuroanatomy of Diagram 1A also shows schematically the neuronalcontroller portion 12 connectivity via various projections P (see arrowsin the Figure) of one embodiment. A projection P can contain multiplesynaptic connections between neuronal units. A synaptic connection canallow a signal to be sent from a pre-synaptic neuronal unit to apost-synaptic neuronal unit. Projections P can be either within a neuralarea or between neural areas. Furthermore, projections P can haveproperties as indicated by the legend in Diagram 1A, which are (1)“voltage independent”, (2) “voltage dependent”, (3) “plastic”, (4)“inhibitory,” and (5) “value dependent” and will be described in detailbelow.

Input to the simulated neural system 12 can come from the CCD camera 16,wheel odometry 18, and IR sensors 22, 24 for detection of walls 30 andhidden platform 36 of environment 28. The neural areas of simulatedneural system 12 can be analogous to the visual cortex (V1, V2/4), theinferotemporal cortex (IT), parietal cortex (Pr), head direction cells(HD), anterior thalamic nuclei (ATN), motor areas for egocentric heading(M_(HDG)), a value system (S), and positive and negative reward areas(R+, R−). The hippocampus can be connected with the three major sensorinput streams (IT, Pr, ATN), the motor system (M_(HDG)), and the valuesystem (S). For clarity, the intrinsic connections within each of theneural areas are omitted from Diagram 1A.

Diagram 1B shows the detailed connectivity within the hippocampalregion. The modeled hippocampus can contain areas analogous toentorhinal cortex (ECIN, ECOUT), dentate gyrus (DG), the CA3 subfield,and the CA1 subfield. These areas can contain interneurons that causefeedback inhibition (e.g. CA3→CA3FB→CA3), feedforward inhibition (e.g.DG→CA3FF→CA3), and rhythmic inhibition (e.g. BF→Hippocampus).

Much more detail of the simulated nervous system 12 is given in Table 1and Table 2 described below. But overall, in one version of thesimulated nervous system 12, there are a total of 50 neuronal areas,90,000 neuronal units within the 50 neuronal areas, and approximately1.4 million synaptic connections.

The neuronal controller portion 12 shown in Diagram 1A can be comprisedof five systems: (1) a visual system 38, (2) a head direction system 40,(3) a hippocampus formation 42, (4) a value system 44, and (5) an actionselection system 46.

Diagram 1A. Visual System 38.

The visual system 38 can be modeled on the primate occipitotemporal orventral cortical pathway and a dorsal cortical pathway. The ventralcortical pathway shown in Diagram 1A, in the visual system(V1_color→V2/4_color→IT), can contain neuronal units in successive areashaving progressively larger receptive fields until, in inferotemporalcortex (IT), receptive fields cover the entire visual field and have notopography. The dorsal cortical pathway shown in Diagram 1A(V1_width→V2/4_width→Pr) can contain neuronal units that respond to thesize and position of objects.

Visual images from the CCD camera 16 can be filtered for color and edgesand the filtered output directly affects neural activity in areaV1-Color and V1-Width. The CCD camera 16 can send, for example, 320×240pixel RGB video images, via an RF transmitter on NOMAD 10, to a framegrabber attached to one of the computer workstations (described below)running the neural simulation. The image can be spatially averaged toproduce an 80×60 pixel image. Different sized Gabor filters (8×8, 16×16,32×32, and 64×64) can be used to detect vertical edges of varyingwidths. The output of the Gabor function can map directly onto theneuronal units of the corresponding V1 width sub-areas (V1-width8,V1-width16, V1-width32, and V1-width64) (not shown in Diagram 1A). Colorfilters (e.g., red positive center with a green negative surround, rednegative center with a green positive surround, blue positive withred-green negative, and blue negative with red-green positive) may beapplied to the image. The outputs of the color filters can be mappeddirectly onto the neuronal units of V1 Color sub-areas V1-red, V1-green,V1-blue, and V1-yellow (not shown in Diagram 1A). V1 neuronal unitsproject retinotopically to neuronal units V2/V4.

Diagram 1A. Head Direction System 40.

Neurons in the areas HD are often called “head direction” cells.Information obtained from the wheels 20 of NOMAD 10 can be used toestimate the current heading of NOMAD 10. This information is input intothe head direction neural area (HD). Each of the 360 HD neuronal units(see Table 1) can have a cosine tuning curve, which responds maximallyto a preferred heading with a tuning width of πradians:cos(HD_(i)−curr_heading);where HD_(i) is a head direction cell with a preferred direction of$\left( {\frac{i}{360}2\pi} \right),$i ranges from 0 to 359, and curr_heading is NOMAD's heading, which iscalculated from odometer information.

The head direction cells can project topographically to an areaanalogous to the anterior thalamic nucleus (see HD→ATN in Table 2 andDiagram 1A) and to a motor area based on heading (see HD→M_(HDG) inTable 2 and Diagram 1A—note, this path not in Diagram 1A). A new headingfor NOMAD 10 can be chosen based on the activity in neuronal areaM_(HDG) (see Diagram 1A—Action Selection, below, for further details).

Hippocampal Formation—Neural Area Hippocampus 42.

The architecture of the simulated hippocampal formation can be based onrodent neuroanatomy. The input streams into the hippocampus can be fromthe associative cortex areas in the simulated nervous system and arriveat the hippocampus via the entorhinal cortex 2 (see ATN→EC_(IN),IT→EC_(IN), Pr→EC_(IN) in Table 2 and FIGS. 8A-8B). The perforant pathcan project mainly from entorhinal cortex to the dentate gyrus but alsoto the CA3 and CA1 subfields (see EC_(IN)→DG EC_(IN)→CA3, EC_(IN)→CA1 inTable 2 and Diagram 1B). The mossy fibers (see DG→CA3 in Table 2 andDiagram 1B), Schaffer collaterals (see CA3→CA1 in Table 2 and Diagram1B), and divergent projections from the hippocampus back to the cortex(see CA1→EC_(OUT)→ATN, IT, Pr) in Table 2 and FIGS. 8A-8B) can also bepresent in the simulated nervous system 12. The prevalent recurrentconnectivity found in the hippocampal formation can also be included inthe simulated nervous system 12 (see EC_(IN)→EC_(OUT) DG→DG, and CA3→CA3in Table 2 and Diagram 1B). All of the connections within thehippocampus proper (i.e., EC_(IN), EC_(OUT), DG, CA3, CA1) can bevalue-independent and plastic (see section Synaptic Plasticity, below).

Unique patterns of intrinisic and extrinsic, feedback and feedforwardinhibitory connections in the hippocampus can play an important role inhippocampus processing. Consequently, the neuronal controller portion 12can include feedback inhibitory connections (see EC→EC_(FB)→EC,DG→DG_(FB)DG, CA3→CA3_(FB)→CA3, CA1→CA1_(FB)→CA1 in Table 2 and Diagram1B) and feedforward inhibitory connections (see DG→CA3_(FF)→CA3,CA3→CA1_(FF)→CA1 in Table 2 and Diagram 1B). These connections can beuseful for separating inputs and network stability.

Basal Forebrain and Theta Rhythm (Table 2).

The simulated basal forebrain (BF) can provide an extrinsic theta rhythmfor the neural simulation. The function of the simulated basal forebrainarea can be to gate input into the hippocampus and keep activity levelsstable. The BF area can have a rhythmic activity over 13 simulationcycles:BF(t)=theta(tmod13);where theta={0.01, 0.165, 0.33, 0.495, 0.66, 0.825, 1.00, 0.825, 0.66,0.495, 0.33, 0.165, 0.01}. BF can project to all hippocampal areas withinhibitory connections (see BF→EC_(IN), EC_(OUT), DG, CA3, CA1 in Table2). The level of inhibition, which is adaptive, can keep the activity inhippocampal regions within specific ranges:Δsf _(r)(t)=(s _(r)(t)−tgt _(r));BF _(r)(t)=BF(t)+sf _(r)(t)where r denotes the region (i.e. EC_(IN), EC_(OUT), DG, CA3, CA1),sf_(r)(t) is the scale factor at time t, s_(r)(t) is the percentage ofactive neuronal units in region r at time t, tgt_(r) is the desiredpercentage of active units in area r (EC_(IN)=10%, EC_(OUT)=10%, DG=20%;CA3=5%, and CA1=10%), and BF_(r)(t) is the pre-synaptic neuronal unitactivity for a BF to hippocampus region r connection.Diagram 1A. Value Systems and Temporal Difference Learning 44.

Activity in the simulated value systems 44 can signal the occurrence ofsalient sensory events experienced by NOMAD 10 and this activitycontributes to the modulation of connection strengths in the pathwaysshown. Initially, value system S can be activated by the IR detector 24that detects hidden platform 36 (see R

→S in Table 2 and Diagram 1A), causing potentiation of value dependentconnections (CA1→S and CA1→M_(HDG)), or by obstacle avoidance the IRdetectors 22 (see R→S in Table 2 and Diagram 1A), causing depression ofvalue dependent connections. After learning, pursuant to the trainingtrials mentioned above and described more fully below, the area CA1 canhave an effect on area S activity. The magnitude of potentiation ordepression can be based on a neural implementation of a temporaldifference (TD) learning rule. ${{TD}(t)} = \left\{ {\begin{matrix}{{{R^{+}(t)} - \overset{\_}{S\left( {t - \tau} \right)}};} & {R^{+} > 0} \\{{\overset{\_}{S\left( {t - \tau} \right)} - {R^{-}(t)}};} & {R^{-} > 0} \\{{\overset{\_}{S(t)} - \overset{\_}{S\left( {t - \tau} \right)}};} & {otherwise}\end{matrix};} \right.$where S(t) is the average activity of the value system S at time t, τ isone theta cycle or 13 simulation cycles, R⁺ is a positive reward andequal to 1 if the downward facing IR detector 24 is triggered, meaningthat NOMAD 10 is over the hidden platform 36, and R is a penalty andequal to 1 if one of the seven IR detectors 22 around NOMAD's base istriggered, meaning that NOMAD 10 is too close to a wall 30. The basicidea of the temporal difference TD rule can be that the learning isbased on the difference between temporally successive predictions ofrewards. In other words, the goal of the learning can be to make thelearner's current prediction for the current input pattern more closelymatch the next prediction at the next time interval (τ). If thepredicted value increases, TD is positive and affected synapticconnections of the simulated nervous system 12 are potentiated, and ifthe predicted value decrease TD is negative and affected synapticconnections are depressed. After some experience, values can increase asNOMAD 10 heads toward the platform 36 and thus reinforce movementstowards the platform 36. Alternatively, values can decrease as NOMAD 10nears obstacles, and thus reinforce movements away from walls 30 andother obstacles. Further details on how the temporal difference isapplied to individual synaptic connections are given below whendescribing Synaptic Plasticity.Diagram 1A. Action Selection and Exploratory Behavior 46.

NOMAD 10 can move forward for 3 theta cycles (39 simulation cycles) andthen selects a new heading, as described below. If NOMAD 10 detects anobstacle, it may reverse its direction for 24 inches, and then turn awayfrom the IR sensor 22 that detected the obstacle. If NOMAD 10 detectsthe hidden platform 36, it may turn counter-clockwise 60 degrees andwait for 3 seconds, then turn clockwise for 60 seconds and wait 3seconds, then another 60 degree clockwise turn and 3 second wait, andfinally turn counter-clockwise returning to its original heading, allunder control of system 12. The simulation ends at this point, and thecurrent state of the simulated nervous system 12 is saved to hard disk.Otherwise, after 3 theta cycles, NOMAD 10 may choose a new heading basedon activity in the motor area (M_(HDG)) of the simulated nervous system12. From its original heading, NOMAD 10 may first turn counter-clockwise60 degrees and wait for 3 seconds, then turn clockwise for 60 degreesand wait 3 seconds, then another 60 degree clockwise turn and 3 secondwait, and finally turn counter-clockwise returning to its originalheading, all under control of system 12. The average activity of M_(HDG)is calculated during the wait periods. A softmax algorithm may be usedto create a probability distribution for choosing a new heading based onthe following equation:${{{p({newhdg})} = \frac{\exp\left( {40\left( \overset{\_}{M_{HDG}({newhdg})} \right)} \right)}{\text{?}{\exp\left( {40\left( \overset{\_}{M_{HDG}(h)} \right)} \right)}}};}\quad$?indicates text missing or illegible when filedwhere newhdg is a possible new heading for NOMAD 10, M_(HDG)(newhdg) isthe average activity of M_(HDG) at a possible new heading, hdg is thecurrent heading, and h has three positions (current heading, currentheading less 60 degrees, current heading plus 60 degrees). After thesimulated nervous system 12 can select a new heading, NOMAD 10 would beoriented to the new heading and then proceed forward for another 3 thetacycles before activating the heading selection process again.Neuronal Units/Dynamics—Generally

A neuronal unit within a neural area V1 Color, V1 Width etc. can besimulated by a mean firing rate model. The state of each unit can bedetermined by a mean firing rate variable (s). The mean firing ratevariable of each unit corresponds to the average activity or firing ratevariable of a group of roughly 100 neurons during a time period ofapproximately 200 milliseconds.

Synaptic connections between neural units, both within and betweenneuronal areas V1 Color, V1 Width, etc. can be set to be eithervoltage-independent or voltage-dependent, and either plastic ornon-plastic. Voltage-independent connections can provide synaptic inputto a postsynaptic neuron regardless of post synaptic state of theneuron. Voltage-dependent connections can represent the contribution ofreceptor types (e.g. NMDA receptors) that require postsynapticdepolarization to be activated.

In other words, a presynaptic neuron can send a signal along its axonthrough a synapse to a post synaptic neuron. The postsynaptic neuron canreceive this signal and integrate it with other signals being receivedfrom other presynaptic neurons.

A voltage independent connection can be such that if a presynapticneuron is firing at a high rate, then a post-synaptic neuron connectedto it via the synapse will fire at a high rate.

A voltage dependent connection can be different. If the postsynapticneuron is already firing at some rate when it receives a presynapticinput signal, then the voltage-dependent connection can cause the postsynaptic neuron to fire more. Since the postsynaptic neuron can beactive, i.e. already firing, this neuron is at some threshold level whenreceiving the input signal. Therefore, this presynaptic connection canmodulate the post synaptic neuron to fire even more. Thevoltage-dependent connection, no matter how active the presynapticneuron is, may have no effect on the postsynaptic neuron if the latterwere not above the threshold value. Thus, the postsynaptic neuron canhave some given threshold of activity to be responsive or modulated by avoltage-dependent synaptic connection.

Neuronal Units.

The mean firing rate (s) of each neuronal unit can range continuouslyfrom 0 (quiescent) to 1 (maximal firing). The state of a neuronal unitcan be updated as a function of its current state and contributions fromvoltage-independent, and voltage-dependent inputs. Thevoltage-independent input to unit i from unit j can be:A _(ij) ^(VI)(t)=c _(ij) c _(j)(t);where s_(j)(t) is the activity of unit j, and c_(ij) is the connectionstrength from unit j to unit i. The voltage-independent postsynapticinfluence, POST_(i) ^(VI), on unit i can be calculated by summing overall the inputs onto unit i:${{{POST}_{1}^{VI}(t)} = {{\varphi\left( {{POST}_{1}^{VI}\left( {t - 1} \right)} \right)} + {\left( {1 - \varphi} \right)\left( {\sum\limits_{l = 1}^{M}{\sum\limits_{j = 1}^{N_{i}}\left( {A_{ij}^{VI}(t)} \right)}} \right)}}};$where M is the number of different anatomically defined connection types(see Table 2), N_(i) is the number of connections of type M projectingto unit i, and φ is the persistence of synaptic input.

The voltage-dependent input to unit i from unit j can be:A_(ij)^(VI)(t) = Φ(POST_(i)^(VI)(t))c_(ij)s_(j)(t), where${\Phi(x)} = \left\{ {\begin{matrix}{0;} & {x < \sigma_{t}^{vdep}} \\{x;} & {otherwise}\end{matrix};} \right.$where σ_(i) ^(vdep) is a threshold for the postsynaptic activity belowwhich voltage-dependent connections have no effect (see Table 1).

The voltage-dependent postsynaptic influence on unit i,POST_(i) ^(VD),can be given by:${{{POST}_{i}^{VD}(t)} = {{\varphi\left( {{POST}_{i}^{VD}\left( {t - 1} \right)} \right)} + {\left( {1 - \varphi} \right)\left( {\sum\limits_{l = 1}^{M}{\sum\limits_{j = 1}^{N_{i}}\left( {A_{ij}^{VD}(t)} \right)}} \right)}}};$

A new activity, s_(i)(t+1), can be chosen based on the sum of thepostsynaptic influences on neuronal unit i:${POST}_{i} = {{\text{?}{POST}_{j}^{VI}} + {\sum\limits_{k = 1}^{N_{VD}}{POST}_{k}^{VD}}}$?indicates text missing or illegible when filed

The new activity for the neuronal unit is the activity level at thenewly chosen phase, which can then be subjected to the followingactivation function:s_(i)(t + 1) = ϕ(tanh (g_(i)(POST_(i) + ω  s_(i)(t)))), where${\phi(x)} = \left\{ {\begin{matrix}{0;} & {x < \text{?}} \\{x;} & {otherwise}\end{matrix};{\text{?}\text{indicates text missing or illegible when filed}}} \right.$where ω determines the persistence of unit activity from one cycle tothe next, g_(i) is a scaling factor, and σ_(i) ^(fire) is a unitspecific firing threshold.

Specific parameter values for neuronal units are given in Table 1: TABLE1 Parameter Values. Specific values of parameters defining properties ofthe example neuronal units of one embodiment are given in Table 1. AreaSize σ-fire σ-vdep ω g V1 (8) 60 × 80 — — — — HD  1 × 360 — — — — R+ 1 ×1 — — — — R− 1 × 1 — — — — BF 1 × 1 — — — — V2/4-color (4) 6 × 8 0.200.10 0.0 1.0 V2/4-width (4) 15 × 20 0.20 0.10 0.0 1.0 IT 30 × 30 0.200.10 0.0 1.0 ITi 15 × 15 0.20 0.10 0.15 1.0 PR 30 × 30 0.20 0.10 0.0 1.0ATN 30 × 30 0.10 0.10 0.50 1.0 ATNi 30 × 30 0.10 0.10 0.15 1.0 M_(HDG) 1 × 60 0.0 0.10 0.0 1.0 M_(HDG)i  1 × 60 0.0 0.10 0.0 1.0 S 4 × 4 0.00.10 0.0 1.0 EC_(IN) 30 × 30 0.10 0.10 0.50 1.0 EC_(IX)i 15 × 15 0.020.10 0.0 1.0 EC_(OUT) 30 × 30 0.10 0.10 0.50 1.0 EC_(O)i 15 × 15 0.020.10 0.0 1.0 DG 30 × 30 0.10 0.10 0.50 0.75 DG_(pg)i 15 × 15 0.02 0.100.0 1.0 CA3 15 × 15 0.05 0.10 0.50 0.75 CA3_(pg)i 8 × 8 0.02 0.10 0.01.0 CA3_(pp)i 15 × 15 0.02 0.10 0.0 1.0 CA1 20 × 20 0.05 0.10 0.50 0.75CA1_(pg)i 10 × 10 0.02 0.10 0.0 1.0 CA1_(pp)i 10 × 10 0.02 0.10 0.0 1.0

As shown in Table 1, areas V1, HD, R+, R−, and BF are input areas andtheir activity is can be based on the camera image, odometry, and IRsensors respectively, Areas V1 and V2/V4 can have 4 sub-areas for color(red, green, blue, and yellow) and 4 sub-areas for varying widthsconsistent with the example of the enclosed environment 28 in whichNOMAD 10 navigates. Table 1 indicates the number of neuronal units ineach area or sub-area (Size). Neuronal units in each area have aspecific firing threshold (σ-fire), and a threshold above whichvoltage-dependent connections can have an effect (σ-vdep), a persistenceparameter (ω), and a scaling factor (g).

Synaptic connections for the neuronal units can be given in Table 2:TABLE 2 Properties of Anatomical Projections. Specific properties ofanatomical projections and connection types of the simulated nervoussystem are given in Table 2. Projection Arbor p c_(ij)(0) type φ η k1 k2V1-color→V2/4-color [ ] 1 × 1 1.00   003, 0.050 VI — 0.00 0.00 0.00V2/4-color→V2/4-color(intra) [ ] 2 × 2 0.40 0.5, 0.6 VD — 0.00 0.00 0.00V2/4-color→V2/4-color(inter) [ ] 3 × 3 1.00 −0.0012, −0.28  VI — 0.000.00 0.00 V1-width→V2/4-width [ ] 1 × 1, 2 × 2, 3 × 3, 4 × 4 1.00 0.008,0.009 VI — 0.00 0.00 0.00 V2/4-width→V2/4-width(intra) [ ] 2 × 2 0.400.5, 0.6 VD — — — — V2/4-width→V2/4-width(inter) [ ] 3 × 3 1.00 −0.012,−0.014 VI — — — — V2/V4→IT non-topo 0.05 0.03, 0.04 VI — — — — IT→IT [ ]1 × 1 1.00 0.08, 0.14 VI — — — — IT→ITi Θ 2, 3 1.00 0.06, 0.08 VI — — —— ITi→IT [ ] 1 × 1 1.00 −0.36, −0.50 VI — — — — V2/V4→Pr [ ] 1 × 1 0.250.25, 0.30 VD — — — — Pr→Pr Θ 4, 6 1.00 −0.06, −0.08 VI — — — — HD→ATN [] 30 × 2 0.20 0.01, 0.02 VI — — — — ATN→ATNi Θ 10, 15 0.25 0.01, 0.02 VI— — — — ATNi→ATN [ ] 1 × 1 1.00 −0.36, −0.50 VI — — — — HD→M_(HDG) [ ] 1× 1 1.00 0.01, 0.01 VI — — — — M_(HDG) → M_(HDG)i Θ 20, 30 0.50 0.10,0.20 VI — — — — M_(HDG)i → M_(HDG) [ ] 1 × 1 1.00 −0.36, −0.50 VI — — —— ATN→EC_(IN) non-topo 0.001 0.40, 0.50 VI — — — — IT→EC_(IN) non-topo0.001 0.40, 0.50 VI — — — — Pr→EC_(IN) non-topo 0.001 0.40, 0.50 VI — —— — EC_(IN)→EC_(OUT) non-topo 0.05 0.40, 0.80 VI — — — —EC_(IN)→EC_(IN)i Θ 2, 3 0.10 0.45, 0.60 VI — — — — EC_(IN)i→EC_(IN) [ ]1 × 1 1.00 −1.90, −1.20 VI — — — — EC_(IN)→DG [ ] 3 × 3 0.10 0.45, 0.60VI 0.75 0.05 0.90 0.45 EC_(IN)→CA3 [ ] 3 × 3 0.05 0.15, 0.20 VI 0.750.05 0.90 0.45 EC_(IN)→CA1 [ ] 3 × 3 0.04 0.30, 0.40 VI 0.75 0.05 0.900.45 EC_(OUT)→ATN non-topo 0.01 0.40, 0.45 VD — — — — EC_(OUT)→ITnon-topo 0.01 0.40, 0.45 VD — — — — EC_(OUT)→Pr non-topo 0.01 0.40, 0.45VD — — — — EC_(OUT)→EC_(IN) non-topo 0.05 0.04, 0.08 VI — — — —EC_(OUT)→EC_(OUT)i Θ 2, 3 0.10 0.45, 0.60 VI — — — — EC_(OUT)i→EC_(OUT)[ ] 1 × 1 1.00 −0.90, −1.20 VI — — — — DG→CA3 [ ] 3 × 3 0.03 0.45, 0.60VI — — — — DG→DG_(FB)i Θ 2, 3 0.10 0.45, 0.60 VI — — — — DG_(VV)i→DG [ ]1 × 1 1.00 −0.90, −1.20 VI — — — — DG→CA3_(FP)i Θ 2, 3 0.10 0.45, 0.60VI — — — — CA3_(FP)i→CA3 [ ] 1 × 1 1.00 −0.90, −1.20 VI — — — — CA3→CA1[ ] 3 × 3 0.08 0.45, 0.60 VI 0.75 0.05 0.90 0.45 CA3→CA3 non-topo 0.100.15, 0.20 VI 0.75 0.05 0.90 0.45 CA3→CA3_(FB)i Θ 2, 3 0.10 0.45, 0.60VI — — — — CA3_(FB)i→CA3 [ ] 1 × 1 1.00 −0.90, −1.20 VI — — — —CA3→CA1_(FP)i Θ 2, 3 0.10 0.45, 0.60 VI — — — — CA1_(FP)i→CA1 [ ] 1 × 11.00 −0.90, −1.20 VI — — — — CA1→EC_(OUT) [ ] 3 × 3 0.25 0.60, 0.75 VI0.75 0.05 0.90 0.45 CA1→M_(ADG)# [ ] 3 × 3 1.00 0.01, 0.02 VD — 0.050.90 0.45 CA1→S* [ ] 3 × 3 1.00 0.01, 0.02 VD — 0.005 0.90 0.45 R+→Snon-topo 1.00 0.25, 0.25 VI — — — — R−→S non-topo 1.00 0.25, 0.25 VI — —— — BF→ EC_(IN)EC_(OUT)DG, CA3, CA1 non-topo 0.05 −0.01, −0.02 VI — — ——

As indicated in Table 2, a presynaptic neuronal unit can connect to apostsynaptic neuronal unit with a given probability (p) and givenprojection shape (Arbor). This arborization shape can be rectangular

with a height and width (h×w), doughnut shaped “Θ” with the shapeconstrained by an inner and outer radius (r1, r2), or non-topographical“non-topo” where any pairs of presynaptic and postsynaptic neuronalunits have a given probability of being connected. The initialconnection strengths (c_(ij)(0), can be set randomly with the rangegiven by a minimum and maximum value (min, max). A negative value forc_(ij)(0), indicates inhibitory connections. Connections marked with“intra” denote those within a visual sub-area and connections markedwith “inter” denote those between visual sub-areas. Projections marked #can be value-dependent. As already mentioned, a connection type can bevoltage-independent (VI), or voltage-dependent (VD). φdenotes thepersistence of the connection. Non-zero values for η, k₁, and k₂ signifyplastic connections.

Synaptic Plasticity—Value Independent Plasticity.

Synaptic strengths can be subject to modification according to asynaptic rule that depends on the phase and activities of the pre- andpostsynaptic neuronal units. Plastic synaptic connections can be eithervalue-independent (see EC_(IN)→DG, CA3, CA1; DG→CA3; CA3→CA1;CA1→EC_(OUT) in Diagram 1B and Table 2 or value-dependent (see CA1→S,CA1→M_(HDG)). Both of these rules are based on a modified BCM learningrule in which synaptic change (Δc_(ij)) is a function of the post- andpre-synaptic neuronal unit's activity and a sliding threshold (θ) asshown graphically in the insert next to Diagram 1B. Synapses betweenneuronal units with strongly correlated firing phases can be potentiatedand synapses between neuronal units with weakly correlated phases aredepressed; the magnitude of change is determined as well be pre- andpostsynaptic activities.

Thus, value-independent synaptic changes in c_(ij) can be given by:Δc _(ij)(t+1)=ηs _(i)(t)s_(j)(t)BCM(s _(i))s_(i)(t) and s_(j)(t) are activities of post- and presynaptic units,respectively, and ηis a fixed learning rate. The function BCM can beimplemented as a piecewise linear function, taking post-synapticactivity as input, which is defined by a sliding threshold, θ, twoinclinations (k₁, k₂) and a saturation parameter ρ (ρ=6 throughout):${{BCM}(s)} = \left\{ \begin{matrix}{{{- k_{1}}s};} & {s \leq {\theta/2}} \\{{k_{1}\left( {s - \theta} \right)};} & {{\theta/2} < s \leq \theta} \\{{k_{2}{{\tanh\left( {\rho\left( {s - \theta} \right)} \right)}/\rho}};} & {otherwise}\end{matrix} \right.$

The threshold can be adjusted based on the post-synaptic activity:Δθ=0.25(s ²−θ)Value-independent plasticity can be subject to weight normalization toprevent unbounded potentiation:${c_{ij} = \frac{c_{ij}}{{sqrt}\left( {\sum\limits_{k = 1}^{K}c_{kj}^{2}} \right)}};$where c_(ij) is a particular connection, and K is the total number ofconnections onto neuronal unit j.Synaptic Plasticity—Value Dependent Plasticity.

The rule for value-dependent plasticity can differ from thevalue-independent rule in the synaptic change is governed by thepre-synaptic activity, post-synaptic activity, and temporal differenceTD from the value systems (see above re Value System and TemporalDifference Learning). The synaptic change for value-dependent synapticplasticity can be given as:c _(ij)(t+1)=ηs _(i)(t)s _(j)(t)TD(t);

-   -   where TD(t) is the temporal difference value at time t.

1. A device comprising: sensors; a controller receiving inputs from thesensors, the controller including a neuronal controller portion and anon-neuronal controller portion; and actuators receiving commands fromthe controller.
 2. The device of claim 1, wherein at least one of theactuators is controlled by the neuronal controller portion and whereinat least another one of the actuators is controlled by the non-neuronalcontroller portion.
 3. The device of claim 2, wherein an actuator of theactuators is controlled by both the neuronal controller portion andnon-neuronal controller portion.
 4. The device of claim 1, wherein anactuator of the actuators is controlled by both the neuronal controllerportion and non-neuronal controller portion.
 5. The device of claim 1,wherein the device is a robot.
 6. The device of claim 5, wherein boththe neuronal controller portion and the non-neuronal controller portionare on the robot.
 7. The device of claim 5, wherein one of the actuatorsadjusts the orientation of a camera and wherein the neuronal portioncontrols an orientation of the camera.
 8. The device of claim 5, whereinone of the actuators adjusts the movement of the robot and wherein theneuronal controller portion and the non-neuronal controller portionshare control of the movement of the robot.
 9. The device of claim 5,wherein the device includes an actuator to manipulate an object, whereinthe manipulation of the object is controlled by the non-neuronalportion.
 10. The device of claim 5, wherein the non-neuronal controllerportion includes a probabilistic localization module, the probabilisticlocalization module providing data to the neuronal controller portion.11. The device of claim 1, wherein the neuronal controller portionincludes neuronal units.
 12. The device of claim 11, wherein theneuronal units do pre-synaptic and post-synaptic processing.
 13. Thedevice of claim 11, wherein the neuronal units have synaptic connectionsto other neuronal units, the synaptic connections being associated witha connection strength.
 14. The device of claim 13, wherein connectionstrengths are modified in plasticity processing.
 15. A devicecomprising: sensors; a controller receiving inputs from the sensors, thecontroller including a neuronal controller portion and a non-neuronalcontroller portion; and actuators receiving control commands from thecontroller, wherein at least one of the actuators is controlled by theneuronal controller portion and wherein at least another one of theactuators is controlled by the non-neuronal controller portion.
 16. Thedevice of claim 15, wherein an actuator of the actuators is controlledby both the neuronal controller portion and non-neuronal controllerportion.
 17. The device of claim 16, wherein the device is a robot. 18.The device of claim 17, wherein both the neuronal controller portion andthe non-neuronal controller portion are on the robot.
 19. A devicecomprising: sensors; a controller receiving inputs from the sensors, thecontroller including a neuronal controller portion and a non-neuronalcontroller portion, the non-neuronal controller portion including alocalization module, the localization module providing data to theneuronal controller portion, and actuators receiving control commandsfrom the controller.
 20. The device of claim 19, wherein the neuronalcontroller portion provides data to the localization module.
 21. Thedevice of claim 19, wherein at least one of the actuators is controlledby the neuronal controller portion.
 22. The device of claim 19, whereinthe device is a robot.
 23. The device of claim 19, wherein thelocalization module is a probabilistic location module.
 24. A computerprogram product comprising a computer useable medium having controllogic stored therein, wherein said control logic when executed causesone or more processors to perform functions of a neuronal controlportion of a brain based device, said control logic comprising: meansfor causing said one or more processors to perform visual functions;means for causing said one or more processors to perform head directionfunctions; means for causing said one or more processors to performhippocampus functions; means for causing said one or more processors toperform value functions; and means for causing said one or moreprocessors to perform action selection functions.
 25. A computer programcomprising a computer useable medium having control logic storedtherein, wherein said control logic when executed causes one or moreprocessors to simulate one or more neuronal units, said control logiccomprising: means for causing said one or more processors to simulateoperation of one or more neuronal units in a visual system; means forcausing said one or more processors to simulate operation of one or moreneuronal units in ahead direction system; means for causing said one ormore processors to simulate operation of one or more neuronal units in ahippocampus formation; means for causing said one or more processors tosimulate operation of one or more neuronal units in a value system; andmeans for causing said one or more processors to simulate operation ofone or more neuronal units in an action selection system.